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Abstract.  This  position  paper  presents  concepts  and  applications  of  an  ontology 
for  level-one  sensor  fusion.  It  covers  platforms,  sensors,  tangible  entities  as  well 
as  intangible  entities,  such  as  concepts.  Relationships  between  entities  are  part  of 
the  ontology.  Verbs  as  well  as  nouns  are  included.  Concepts  such  as  reasoning 
under  uncertainty  also  are  part  of  the  sensor-fusion  ontology.  Applications  relate 
to  knowledge  discovery  and  the  pattern  recognition  of  potential  security  threats. 

1  Introduction 

Sensor-data  fusion  is  divided  into  four  levels  of  increasing  situation  complexity. 
Level-one  sensor  data  fusion,  which  concerns  object  refinement,  is  defined  as  the  fu¬ 
sion  of  data  related  to  detection,  tracking,  classification  and  the  identification  of  plat¬ 
forms  such  as  ships  and  aircraft  without  consideration  to  the  intent  of  the  piatforms. 
(See,  for  exampie,  [1]).  Levei  two  is  focused  on  situation  refinement  in  which  the  re- 
iationship  between  piatforms  becomes  important.  Levei  three  is  about  threat  refine¬ 
ment  and  addresses  the  intent  of  hostiie  piatforms.  Levei  four  addresses  process  re¬ 
finement  in  which  a  commander  tries  to  predict  hostiie  actions.  Knowiedge  discovery 
and  the  fusion  of  sensor  data  with  information  from  a  variety  of  other  observations 
can  assist  miiitary  and  iaw-enforcement  efforts  to  detect  anomaious  behavior  of  piat¬ 
forms,  thus  contributing  to  security  and  threat  detection  at  seaports  and  airports. 

A  comprehensive  ontoiogy  for  ievei-one  sensor  fusion  inciudes  severai  sub-ieveis 
of  ontoiogy  and  severai  different  dimensions  of  ievei-one  fusion:  piatforms  and  sen¬ 
sors,  characteristics,  tangibie  and  intangibie,  nouns  and  verbs,  reiationships  between 
variabies,  concepts  such  as  data  combinations.  For  exampie,  the  speed  of  a  ship  and 
its  position  are  to  a  first  approximation,  independent  data.  Knowing  oniy  the  speed  of 
a  ship  may  not  trigger  an  aiert.  Simiiariy,  its  position  aione  may  not  be  significant. 
However,  the  knowiedge-discovery  process,  may  reveai  that  the  speed  of  a  particuiar 
ship  iocated  in  a  certain  area  is  unusuai  couid  signify  iiiegai  activity. 

Data  sources  on  a  sensor  ontoiogy  inciude  data  dictionaries  that  contain  terms  as¬ 
sociated  with  muitipie  sensor  types  such  as  acoustic,  magnetic,  visuai,  imagery,  eiec- 
tro-opticai  etc.,  as  weii  as  other  sources  that  describe  how  sensors  work  aione  and  to¬ 
gether,  such  as  is  the  case  with  sensor  fusion.  This  paper  defines  concepts  and  speci¬ 
fies  reiationships  between  entities  for  the  objects  and  concepts. 

The  miiitary  and  iaw-enforcement  organizations  need  a  singie,  integrated,  iogicai 
and  nationai  sensor  ontoiogy  to  support  knowiedge  bases  in  expert  systems  designed 
for  joint  use  and  for  homeiand  security.  It  represents  the  future  for  fusion,  sensor 
networks  and  inteiiigence.  Existing  ontoiogies,  iike  databases,  are  fragmented,  in- 
compiete  and  in  different  formats.  This  work  was  performed  as  part  of  a  project  at 
SSC-SD  to  test  and  evaiuate  Buiiding  the  Singie  Integrated  Picture  (BSIP)  [2]. 
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2  Symbology  and  levels  of  Abstraction 

Ontologies  themselves  are  an  abstraction  from  physical  objects  and  actions.  Ontolo¬ 
gies  handle  multiple  levels  of  abstraction  in  a  hierarchical  manner.  Sensor  fusion  has 
a  particular  challenge  in  that  it  involves  a  meta-abstraction.  For  example,  data  about 
platforms  and  sensors  are  an  abstraction  from  physical  platforms  and  sensors.  The 
data  are  represented  as  different  icons  in  decision-support  systems  for  different  user 
communities.  This  second  level  of  abstraction  is  a  meta-abstraction. 

Every  level  of  abstraction  introduces  additional  error  modes,  especially  with  level- 
one  fusion.  To  reduce  difficulty  and  confusion,  information  exchange  needs  to  take 
place  at  the  same  level  of  abstraction.  For  example,  user  communities  should  com¬ 
municate  data  rather  than  specify  the  conventions  used  to  represent  data  on  displays. 
A  symbol  that  is  a  yellow  in  one  community  is  white  in  another. 

The  symbology  of  a  community  depends  on  the  assumptions  that  the  community 
makes  about  the  data  and  platforms  that  the  symbols  are  supposed  to  represent. 
Communities  that  specialize  in  force  planning  operate  in  an  analytical  mode  and  pre¬ 
fer  complex  symbols  that  are  designed  to  communicate  relatively  large  quantities  of 
information  at  a  glance  to  personnel  who  know  how  to  interpret  them. 

In  contrast,  in  a  user  community  that  deals  with  tactical  operations  in  which  deci¬ 
sions  need  to  be  made  in  time-critical  scenarios,  the  symbols  tend  to  be  simpler.  For 
example,  a  hostile  submarine  is  represented  simply  as  a  red,  inverted  “V”  with  a  dot 
in  the  middle  of  it  to  show  the  assumed  position.  This  symbol  system  is  designed  to 
alert  an  operator  to  danger,  not  to  facilitate  long-term  planning  [3]. 

An  ontology  that  supports  level-one  sensor  fusion  includes  concepts  from  the 
planning  and  operational  communities.  Different  groups  conceptualize  and  use  sen¬ 
sor  data  in  different  ways.  Knowledge  discovery  for  detecting  complex  patterns  in 
the  battle  space  or  during  peace-time  security  missions  will  need  to  account  for  the 
different  levels  of  granularity  and  detail  that  are  used  in  planning  and  operations. 

3  Relationships  between  Entities 

Nouns  represent  the  entities  and  verbs  contribute  to  the  relationships  between  them. 
For  example,  some  important  relationships  include  basic  verbs  such  as  “is  a,’’  “is  a 
kind  of,’’  “has  a,’’  “is  between,’’  “originates  from’’  and  “detects.’’  In  addition  to  verbal 
relationships,  other  interactions  such  as  the  speed-direction  interaction  mentioned 
above,  and  the  speed-location  interaction  also  can  be  important  in  a  knowledge- 
discovery  task.  In  general,  a  complete  ontology  also  supports  concepts  based  on  in¬ 
verse  relationships,  such  as  “If  A  sends  C  to  B,  C  must  have  been  at  one  time  be¬ 
tween  A  and  B.’’  In  addition  to  the  general  verbs  used  in  any  ontology,  certain  verbs 
apply  especially  to  sensors.  The  verb,  “detect,’’  is  quite  basic  to  sensors.  Other  verbs 
are  “identify,’’  “classify,’’  “occlude,’’  “attenuate,’’  “amplify’’  and  “fuse.’’ 

Semantic  relationships  between  actions  play  an  important  and  often  key  role  in 
sensor-related  concepts.  Finer  grained  relationships  are  also  possible,  such  as 
strength,  enablement  and  temporal  information  [4].  For  example,  X  may  happen  be¬ 
fore  Y.  X  could  be  detect  and  Y  could  be  fuse.  In  this  case  X  enables  Y  because  Y 
cannot  occur  with  out  first  having  X.  It  also  implies  that  X  must  occur  BEFORE  Y, 
thus  invoking  the  concept  of  temporal  information  being  stored  in  verb  usage.  Any 


comprehensive  knowledge  discovery  engine  must  process  relationships  between  enti¬ 
ties  and  the  verbs  that  are  part  of  and  that  influence  these  relationships. 

An  example  of  strength  relations  in  sensor  verbs,  is  the  fact  that  “identify”  is 
stronger  than  “classify,”  which  in  turn  is  stronger  than  “detect.” 

“Happens  before”  is  another  semantic  relation  [4].  For  example,  when  referring  to 
a  sensor-data-acquisition  task,  “acquire”  happens  before  “store.”  Also,  platform  “de¬ 
tection”  happens  before  “classification”  or  “identification.”  Thus,  “detect,”  “clas¬ 
sify,”  and  “identify,”  are  related  not  only  by  verb  strength  but  also  by  the  “happens 
before”  semantic  relation.  These  concepts  are  described  in  section  7. 

Antonymy  [4]  also  is  present  in  sensor  concepts.  For  example,  “amplify”  is  the  op¬ 
posite  of  “attenuate”  when  used  in  reference  to  signals. 

In  addition  to  verbs,  many  more  nouns  apply  to  an  ontology.  For  example  the  noun 
ontology  includes  the  following  terms  and  the  relationships  between  them:  geometry, 
detector,  platform,  range,  signal,  sensor,  mode,  fusion,  uncertainty,  and  error.  The 
next  sections  cover  some  of  these  concepts  in  more  detail. 

4  Platforms  -  Concepts  and  Hierarchy 

The  general  category  of  platforms  can  be  divided  along  several  dimensions.  Among 
them  are:  stationary  or  capable  of  motion;  machine  powered  and  wind  powered;  air¬ 
craft,  vessels,  land  vehicles  and  amphibious  vehicles;  modern  and  archaic;  driven  by 
humans  vs.  autonomous.  The  main  feature  that  all  platforms  have  in  common  in  the 
context  of  target  detection  is  that  all  platforms  are  capable  of  motion  or  at  least  they 
were  capable  of  motion  in  the  recent  past.  Moreover,  a  platform  must  be  capable  of 
transporting  another  entity,  such  as  humans,  a  payload,  cargo,  or  any  combination  of 
these.  This  definition  automatically  excludes  ships  and  aircraft  that  have  been 
brought  onto  land  in  a  display  status  and  are  no  longer  functional  for  transportation 
and,  therefore  are  of  little  to  no  interest  in  terms  of  target  detection. 

The  characteristics  and  performance  of  platforms  constitute  the  main  source  of  in¬ 
formation  from  which  platforms  are  detected,  classified  and  identified.  Thus,  a  plat¬ 
form  ontology  will  have  two  main  branches,  one  for  nouns  (what  it  is  and  has)  and 
the  other  for  verbs  (what  it  does).  For  target  detection,  an  essential  feature  of  plat¬ 
forms  is  that  they  can  be  divided  into  parts,  both  conceptually  and  physically.  The 
meronomy  of  a  platform  for  signal  detection  purposes  includes  those  particular  parts 
that  give  rise  to  signals  that  can  reveal  the  existence  of,  classify,  or  identify  the  plat¬ 
form.  The  “partology”  of  a  platform  is  an  important  part  of  signal-detection  ontology. 

Performance  is  a  good  source  of  information  on  platforms,  but  performance  alone 
often  cannot  provide  enough  information  to  complete  a  target-identity  determination. 
Information  about  a  platform  can  be  revealed  by  its  velocity,  which  consists  of  two 
parts,  speed  and  direction.  The  speed  of  a  platform  can  reveal  the  class  to  which  it 
belongs  by  excluding  any  platform  with  a  maximum  speed  less  than  the  observed 
speed  of  the  platform.  The  direction  of  motion  also  can  reveal  certain  information 
about  the  platform.  The  observed  speed  and  location  of  platform  along  with  its  no¬ 
tional  capabilities  (e.g.  top  speed,  operating  depth,  minimum  depth  required  for  draft, 
etc.)  together  may  provide  enough  information  to  classify  a  platform. 


5  Signals 


For  level-one  fusion,  signals  are  emissions  from  platforms  that  have  a  certain  form  of 
energy,  such  as  acoustic,  magnetic,  electro-optic,  radar,  etc.  This  energy  is  character¬ 
ized  by  several  features,  such  as  peak  frequency,  if  pulsed,  pulse  repetition  rate, 
speed  (which  varies  with  signal  type  and  medium),  spectral  characteristics,  onset,  and 
duration.  Signals  can  result  from  passive  detection,  as  is  the  case  with  magnetic  sig¬ 
natures,  or  from  active  sonar  or  radar,  in  which  another  entity,  usually  another  plat¬ 
form  supplies  the  initial  source  of  energy  and  the  reflection  of  this  emission  which  is 
altered  by  the  platform  of  interest  is  analyzed.  The  assumption  on  which  tasks  of  de¬ 
tection,  classification  and  identification  are  based  is  that  by  analyzing  signals,  includ¬ 
ing  visual  signals  analyzed  directly  by  the  human  nervous  system,  one  can  determine 
required  information  (within  a  particular  timeframe)  about  the  platform  that  was  the 
source  or  reflection  of  these  signals.  In  many  cases  this  is  a  tme  assumption.  In  oth¬ 
ers,  too  much  uncertainty  prevents  a  timely  determination. 

The  following  concepts  are  a  part  of  the  signal  ontology:  emission,  reflection,  en¬ 
ergy  type  (e.g.  acoustic  or  electromagnetic),  energy  characteristics  like  spectral  fea¬ 
tures  (e.g.  frequency,  pulse  repetition  rate,  inter-pulse  interval,  amplitude,  wave 
speed  and  wavelength),  imagery  and  its  patterns  and  features,  and  reasoning  under 
uncertainty.  Other  important  concepts  to  be  included  in  a  signal  ontology  are  signal 
jamming,  signal  processing,  noise,  signal-to-noise  ratio,  and  detection. 

6  Sensors 

A  sensor  as  a  device  that  responds  to  a  physical  stimulus,  (such  as  heat,  light,  sound, 
pressure,  magnetism,  or  a  particular  motion)  and  transmits  a  resulting  impulse  for 
measurement  or  operating  a  control.  In  level-one  fusions,  sensors  detect  signals  from 
platforms.  Sensors  can  be  mechanical,  electrical,  electro-optical,  chemical,  or  bio¬ 
logical  (part  of  a  living  organism).  Sensors  can  be  land  based  or  platform  based,  such 
as  towed  arrays.  The  ontology  includes  concepts  such  as  signals,  detection,  modes  of 
operation,  detection  threshold,  tasking  of  sensors,  sensor  arrays,  operational  limita¬ 
tions  (e.g.  range,  depth,  distance  and  altitude),  swath,  resolution,  passive  sensors  (e.g. 
hydrophone  array),  and  active  sensors  (e.g.  sonar).  Concepts  that  deal  with  the  inter¬ 
action  of  signals  with  sensors  include  detection  and  signal-to-noise  ratio  mentioned 
in  the  section  on  signals.  Thus,  depending  on  the  ontological  representation  either 
multiple  paths  will  link  these  concepts  or  they  will  occur  twice  in  the  ontology. 

Sensor  geometry  is  also  important,  as  some  sensors  are  not  omni-directional.  Their 
geographical  environment  limits  others.  Thus  concepts  that  apply  to  geometry  in 
general,  such  as  points,  lines,  line  segments,  polygons,  circles,  ellipses,  and  other 
spatial  entities  are  also  a  part  of  the  ontology  that  applies  to  sensors  in  particular. 
Some  examples  of  spatial  relationships  and  geometry  as  applied  to  sensors  are  as  fol¬ 
lows:  1.  An  ellipse  is  drawn  around  a  position  to  indicate  the  error  in  the  position 
measurement.  The  major  and  minor  axes  of  the  ellipse  represent  the  uncertainty  in 
the  coordinates  of  the  position.  2.  If  a  sensor  must  detect  a  target,  the  target  must  be 
within  the  range  of  the  sensor,  both  in  the  distance  (radius)  and  angularly  (azimuth). 
Three-dimensional  geometry  is  important  when  a  sensor  is  located  in  a  canyon  and  it 
depends  upon  a  line-of-sight  geometry  to  detect  a  target. 


7  Data  Fusion  and  Tracking 

The  branch  of  the  ontology  that  deals  with  data  fusion  and  tracking  includes  the  fol¬ 
lowing  concepts:  sensor  data  fusion,  fusion  algorithms,  track  fragmentation,  data  per¬ 
ishability  detection  vs.  classification,  and  identification  vs.  classification.  Data  fusion 
includes  the  concept  of  correlating  multiple  observations  of  the  same  platform  with 
different  sensors,  or  with  the  same  sensor  at  different  times  along  the  platform’s  tra¬ 
jectory.  This  is  known  as  tracking.  When  something  intermpts  the  tracking  process 
this  may  result  in  track  fragmentation.  There  is  no  guarantee  that  all  track  fragments 
that  originate  from  the  observation  of  a  given  platform  will  be  identified  as  having  an 
association  with  that  platform. 

The  notion  of  data  perishability  invokes  the  concept  of  time  sensitivity.  Time  criti¬ 
cality  is  discussed  in  [5].  This  concept  relates  to  the  idea  that  data  that  are  useful  now 
may  not  be  of  any  utility  in  the  near  future,  when  the  situation  can  change  rapidly  as 
platforms  move  in  and  out  of  range  of  sensors.  Time-sensitive  data  are  useful  in  an 
operational  sense  only  as  long  as  a  tactical  action  can  be  performed  in  which  the 
knowledge  of  these  data  plays  a  significant  role.  After  a  certain  time  period  has  ex¬ 
pired,  these  data  may  remain  useful  for  the  purposes  of  archives  and  strategic  analy¬ 
sis,  but  the  operational  picture  has  changed  significantly  that  would  render  these  data 
irrelevant  for  operational  applications. 

Detection  is  the  act  of  becoming  aware  of  the  presence  of  a  platform  or  a  target  is 
known.  Classification  occurs  when  the  platform  is  put  into  a  known  category.  Identi¬ 
fication  occurs  when  the  platform  can  be  named,  or  its  serial  number  becomes  known 
and  all  other  platforms  can  be  excluded,  even  those  in  the  same  class.  These  concepts 
occur  in  the  ontology  as  a  sequence.  Identification  and  classification  include  detec¬ 
tion,  as  detection  must  occur  before  a  platform  can  be  classified. 

Tracking  involves  uncertainty,  which  is  part  of  the  sensor  ontology  covers  the  fol¬ 
lowing  concepts:  epistemological  concepts,  percent  probability,  hypothesis  forma¬ 
tion,  hypothetical  platform,  proposed  ship  identity,  detection  vs.  classification,  identi¬ 
fication  vs.  classification,  confirming  evidence,  and  error.  For  example,  do  I  see  one 
platform  as  several  tracks  with  disjoint  multiple  parts  from  various  sensors,  or  do 
these  disjoint  tracks  represent  more  than  one  target? 

8  Knowledge  Discovery  in  Sensor  Data  -  Future  Research 

A  sensor  ontology  can  contribute  value  to  the  knowledge-discovery  process  in  sensor 
data.  Vast  quantities  of  data  that  were  collected  over  the  years  from  sensors  during 
battles  and  exercises  have  fallen  by  the  wayside  because  no  one  has  had  the  tools,  the 
resources,  or  the  inclination  to  interpret  their  significance.  Without  better  data  or¬ 
ganization  and  advanced  tools,  such  as  those  based  on  artificial  intelligence,  the 
problem  has  been  too  hard  to  solve.  Patterns  hidden  in  the  data  could  be  revealed 
much  more  efficiently  if  these  data  were  organized  into  knowledge  based  on  a  sensor 
ontology.  Thus,  the  data  could  be  subject  to  artificial  intelligence  tools  for  knowledge 
discovery  that  otherwise  could  not  be  used  due  to  various  constraints  [5,  6]. 

For  example,  knowledge-discovery  tasks  include  association,  sequential  patterns, 
classification,  and  clustering  [6].  Knowledge  of  the  sensor  ontology  is  a  critical  phase 


of  the  knowledge-discovery  process  because  it  will  prevent  associating  entities  that 
are  semantically  distant  but  lexically  close  or  identical.  For  example,  the  geometry  of 
a  sensor-platform  configuration  may  be  measured  in  nautical  miles,  abbreviated  NM. 
The  wavelength  of  electromagnetic  radiation  measured  from  a  platform  can  be  ex¬ 
pressed  in  nanometers,  also  abbreviated  NM.  The  association  of  a  geometry  with  a 
wavelength  only  on  the  basis  that  they  are  both  abbreviated  NM  is  an  erroneous  re¬ 
sult.  However  in  the  absence  of  a  sensor  ontology  that  would  clearly  separate  the  two 
concepts,  some  knowledge  discovery  software  may  be  inclined  make  the  association. 

A  sequential-pattern  detection  data-mining  task  is  particularly  well  suited  to  the  re¬ 
construction  of  tracks  from  a  database  in  multi-sensor  correlation  and  tracking.  This 
technique  can  be  used  to  analyze  the  correctness  of  past  track  identifications  that  may 
have  been  the  result  of  a  few  readily  available  techniques  in  operational  scenarios. 
With  more  powerful  knowledge-based  techniques,  pattern  detection  can  be  revisited 
on  data  sets  from  past  exercises,  etc.  The  knowledge  base  will  be  based  on  the  sensor 
ontology.  Pedigree  data  also  are  important  here.  Since  records  from  different  sources 
need  to  be  related,  these  records  need  to  be  tagged  with  the  identity  of  the  entity  that 
they  represent  [6].  As  stated  above,  classification  is  not  only  a  level-one  sensor- 
fusion  task,  but  also  a  data-mining  task.  Platform  classifications  that  were  missed  in 
the  operational  environment  due  to  time  and  other  resource  constraints  can  be  dis¬ 
covered  in  an  analytical  mode  after  the  battle  or  exercise  during  a  data-mining  classi¬ 
fication  task  aimed  at  evaluating  the  efficiency  of  the  operational  data  classification 
tools.  A  sensor  ontology  can  help  to  validate  similarity  and  differences  of  entities. 
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